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ABSTRACT 

The research descritjed in this paper was designed to 
assess how evaluative data were used in instructional decision making 
in the Portland (Oregon) Public Schools. Usir^g the Level of Use of an 
Innovation (LoU) interview ^ a component of the Concerns-Based 
Adoption Model (CBAM) , along with data from the Portland Achievement 
Levels Test (PALT) , the study assessed the relationship between 
evaluation use and student achievement, the correlation between 
high-achieving schools and the frequency and effectiveness of 
evaluations, and the factors of evaluation that link it with improved 
school achievement. Thirty-six principals and 82 teacheis in grades 
3-8 were randomly selected for interview by 9 certified LoU 
interviewers. The interviews* focused on four key components of the 
use of PALT evaluat:.ve data: (1) the use of student achievement 
reports? (2) the use of school administrative reports; (3) the role 
of evaluation reports in decision making? c.nd (4) the decision 
situations where evaluation reports are used. Results are reported 
and tabulated for each of the four research questions. The use of 
evaluation was found to have a significant effect on 93 percent of 
the 126 teachers and adr .nistrators interviewed? these findings 
suggest that inservice support is needed to develop knowledge and 
skills in evaluation use, that personality traits may be a factor in 
movement of individuals to higher levels of use, and that the Level 
of Use interview technique produces rich descriptive information on 
how individuals use evaluative data in instructional planning and 
decision making. A bibliography is included and instrumentation is 
appended. (TE) 
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IMPROVING INSTRUCTIONAL DECISION MAKING: 
The Relationship between Level of Use 
of Evaluation and Student Achievement 

Stephanie Mitchell 
Department of Research and Evaluation 
Portland (OR) Public Schools 



Among techniques designed to improve education, there has 
been a recent surge of interest in encouraging the use of 
evaluative data in instructional planning and decision making. 
The basic assumption underlying these efforts is that the use of 
evaluative data will enable teachers and administrators to become 
better decision makers about delivering instruction. This should 
result in improved student achievement. 

The research effort described in this paper was designed to 
help teachers make better use of evaluation information. This 
paper assesses how evaluative data were used in instructional 
decision making. It investigates the level of use of evaluation 
and the relationship to student achievement. Lessons learned in 
this research are discussed in an effort to provide guidance to 
other researchers interested in evaluation use and its impact on 
student achievement. 

Backgro und and Assumptions 

By way of background, this study of evaluation use in 
instructional planning and decision making began in 1S86 as part 
of the Evaluation Department's commitment to the improvement of 



the Portland Achievement Levels Test (PALT) . The PALT program 
was originally developed in 1977 to measure student achievement 
on the District curriculum. All students in grades three through 
eight are tested each fall and spring. The program currently 
disseminates 13 student achievement or administrative evaluation 
reports to the schools twice a year after testing. 

The Evaluation Department produces evaluation reports to 
meet general district, as well as school, needs for evaluative 
data. Teachers and principals use the evaluative information to 
target instruction on specific skills, to group students, to plan 
large and small group instruction, and to communicate with 
parents and other school professionals. The researchers 
emphasize that the PALT is used in instructional decision making, 
not as the sole authority on student achievement, but in 
conjunction with other types of classroom assessment data 
available to Portland's educators. 



Understanding how educators use evaluative data is the 
research focus of the Portland Public Schools' instructional 
decision making study. The study asks the following questions: 

1. What is the relationship between evaluation use 
and student achievement? 

2. Do high achieving schools use evaluative data more 
effectively or more frequently than lower 
achieving schools, when other important variables 
are held constant? 

3. If evaluation use makes a difference, what factors 
link it with improved school achievement? 



The primary research tool used in seeking answers to the 
study questions was the Level of Use of an Innovation (LoU) 
interview, a component of the Concerns-Based Adoption Model 
(Hall, Wallace, and Dossett, 1973). 

The Concerns-Based Adoption Model (CBAM) is a conceptual 
framework for understanding the process of implementing change in 
schools. Research suggests that as incUviduals are introduced to 
change, they pass through predictable phases in their behavior or 
use of an innovation from nonuse, through mechanical use, to 
renewal. While not strictly an ordinal scale, the Levels of Use 
scale usually finds individuals progressing from lower to higher 
levels, although they may move up or down the levels as their use 
of the innovation changes. 

Though the Concerns-Based Adoption Model is well accepted in 
change process literature, its use has been limited in research 
on evaluation use. The LoU technique has been used successfully 
to study evaluation use in other school districts (Pechman and 
King, 1986) . These researchers used the Levels of Use framework 
to define their Structnre of School Evaluation Use (SSEU) . Hall 
(1982) proposed a configuration of key components for evaluation 
use as an innovation. 

The research on student achievement obtained by teachers at 
different Levels of Use is also limited. Loucks (197 5) found a 
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curvilinear relationship with higher student achievement in 
reading correlated with higher use of individualized reading by 
teachers. A different curvilinear relationship was observed in 
math with the same teachers. Reidy and Hord (1979) founc" that 
with an achievement monitoring math program the more extensively 
taachers used the program, the more the students achieved. The 
current study applied the LoU method, specifically Levels of Us2 
of evaluation, to explore its impact on student achievement. 

Levels of Use of Evaluation 

Measure - In order to determine a baseline measure of evaluation 
use, the researchers used a t^cused Levels of Use interview 
technique. The Levels of Use (LoU) model was developed by Gene 
Hall and cjusan Loucks at the Research and Development Center for 
Teacher Education, University of Texas-Austin. 

The LoU interview is a diagnostic protocol useful in describing 
the implementation of an innovation. The protocol is generic; it 
can describe different innovations by changing the point of 
reference. For our purpose, the innovation was defined as use of 
the Portland Achievement Levels Test (PALT) evaluation reports. 
While this is a complex innovation, the reports have been used by 
district staff for ten years. The model identifies eight levels 
of use of the innovation: nonuse, orientation, preparation, 
mechanical use, routine, refinement, integration, and renewal. 
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Figure 1 presents the developmental Levels of Use for the 
innovation of evaluation report use and the behavioral indicators 
as defined in the study. 



Figure 1 

Levels of Use and Behavioral Definitions 



Level of Use 



Behavioral Definition of Evaluation Use 



0 NONUSE 

1 ORIENTATION 

II PREPARATION 
ITI MECHANICAL USE 



IVA ROUTINE 



IVB REFINEMENT 



V INTEGRATION 



VI RENEWAL 



User has little or no knowledge of evaluation 
reports and no involvement with reports. 

User has recently acquired information about 
evaluation reports or is exploring the value 
and demands of evaluation on the user system. 

User prepares for first use of evaluation 
reports . 

User focuses on short term use of evaluation 
reports with little reflection. Changes in 
use are made to meet user needs. User is 
engaged in a step-by-step attempt to master 
the task of using evaluation reports, often 
resulting in disjointed and superficial use. 

Stable use of evaluation reports. Few 
changes are made in use. Little thought is 
giv^n to improving evaluation use or its 
consequences. 

User varies the use of evaluation reports to 
increase impact on students Variations are 
based on knowledge of both short and long- 
term consequences for students. 

User combines own efforts to use evaluation 
reports with activities of colleagues to 
achieve a collective impact on students. 

User reevaluates the quality of evaluation 
reports,^ seeks major modifications or 
alternatives to current reports to increase 
student impact, examines new developments in 
the field, and goal for self or the system. 
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The dependent variable, student achievement, was measured by 
the growth exhibited in reading, math, and language usage palt 
scores achieved by students of the interviewed teachers between 
the fall 1986 and spring 1987 testings. 

^^"'P^^ ~ Levels of Use interviews were conducted with 126 
teachers and administrators by nine certified LoU interviewers. 
Participants included 8 Directors of Instruction, 36 principals, 
and 82 teachers in grades 3-8. Subjects were randomly selected 
from 40 or half of the elementary schools in the district. The 
experience of subjects in their role ranged from 1-25 years and 
their knowledge of evaluation varied from nonuser to specialist. 

Experience shows that asking more removed sources, such as 
superintendents or area administrators, about the evaluation use 
by their staff is questionable. The only way to know if, and 
how, the innovation is being used is to directly assess 
individual use. The Levels of Use method let us focus on how 
individuals were currently using evaluation reports. 

Procedure - Before conducting the LoU interviews, the researchers 
identified key elements of the innovatic-i, use of PALT evaluative 
data. After reviewing preliminary interviews with principals and 
test coordinators and discussing the use of PALT reports with the 
Evaluation Department's management team, four key components of 
the use of PALT evaluative data were identified: 1) the use of 
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student achievement reports, 2) the use of school administrative 
reports, 3) the role of evaluation reports in decision making, 
and 4) the decision .situations where evaluation reports are used. 
An innovation configuration checklist was developed to array the 
various patterns of evaluation report use from ideal use to 
nonuse (see appendix) . 

To measure Levels of Use, interviewees were asked a series 
of branching interview questions and follow-up probes. Interviews 
averaged 20 minutes and appeared to the interviewees to be casual 
conversations about how the individual was using palt reports. 

The Level of Use interviews were conducted by trained LoU 
interviewers and raters. Interviews were tape recorded and the 
resultant tapes were rated by the interviewer. In order to 
maintain reliability, a second rater listened to each tape and 
independently evaluated the interviewee's level of use of 
evaluation reports. If the two raters did not agree on the LoU, 
a third rater listened to the tape. Overall LoU was determined 
when two raters agreed on the rating. 

Of the total 126 LoU tapes, 106 or 84% were agreed on by the 
first two raters. Of the remaining 20 tapes, 16 or 13% were 
classified by the third rater. Only 4 or 3% required a fourth 
rater for a final rating. Overall inter-rater reliability was 
.82 on the Level of Use of evaluation reports. 



Results 



At the request of the Levels of Use interviewer cadre, the 

researchers defined the users and use of evaluation reports as: 

Use of evaluation reports = Teachers, principals, 
or Directors of Instruction who make any use of 
the Portland Achievement Levels Test reports to 
guide decision making in planning or instrucrion 
for students, teachers, or parents. 

Table 1 presents the distribution of the sample by Level of 
Use of PALT evaluation reports. Ninety-three percent of those 
interviewed were identified as users of evaluation reports; seven 
percent of the district sample were nonusers of evaluation data. 



Table 1 

Distribution of the Sample by Level of Use 

(N=126) 



LoU 


0 


I 


II 


III 


IVA 


IVB 


V 


VI 


N 


3 


2 


4 


40 


51 


16 


10 


0 


% 


2% 


2% 


3% 


32% 


40% 


13% 


8% 


0% 



Figure 2 presents the Levels of Use of evaluative reports by 
school role. The LoU interviews found that overall more 
principals (97%) than teachers (85%) reported using evaluation 
data in instructional planning and decision making. 
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Figure 2. Level of Use by School Role 

Use of PALT Evoluaticn Roporta 

0. 




Princlpola 



n ui rvA IVB 
Level of Use 



Among users of eva?uation reports, 32% are Mechanjcal Users 
(LoU ITI) . This group needs training and support in the use and 
interpretation of evaluative data. The largest group, 40% of the 
sample, were identified as Routine users (LoU IVA) . These 
individuals are stable, comfortable users of PALT evaluative 
data. Knowledgeable Routine use is the goal we want people to 
achieve. Interviews also identified a gioup of "enhanced users," 
i.e., individuals at the higher Levels of Use of Refinement (LoU 
IVB) , Integration (LoU V) , or Renewal (LoU VI) . These "enhanced 
users," 21% of the sample, are comfortable with using PALT 
reports and often make refinements to increase student impact. 
Principals are more likely to be higher level users of evaluation 
than teachers; while 36% of the principals are "enhanced users" 
of the reports, only 13% of teachers are among this group. 
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student Achip-vement 

The paragraphs below respond directly to the research 
questions addressing the relationship between levels of use of 
evaluation reports and student achievement. 

Q uestion 1 . what is the relationship between evaluation use and 
student achieveiaent? 

Table 2 presents the means and standard deviations for 
reading growth scores of students by teacher Level of Use. 
Results of a test of significance among the means in reading 
growth are presented in Table 3. The results of -the analysis of 
variance indicated no significant differences among the means in 
growt*- in reading achievement. 

Table 2 

Means and Standard Deviations of Student Reading Growth 

by Teacher Level of Use 
Grades 3-8 



Lou Groups N Mean 

Growth 



13 -.4448 
^^-A 18 .7670 

IV-B 1 -.5776 

V 0 
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standard Deviation 
1.4055 
2.6579 
0.0 



Sources 



LoU 

Error 

Total 



Table 3 

Analysis of Variance of Reading Growth 
by Teacher Level of Use 



SS 



11.081 
143.925 
155.006 



df 



1 
29 
30 



MS 



11.081 
4.963 
5.167 



^.233* 



* p > .05 

Table 4 presents the means and standard deviations for math 
growth scores of students by teacher LoU. Results of a test of 
significance among the means in math are displayed in Table 5. 
The results of the analysis of variance indicated no significant 
differences among the means in mathematics achievement. 



Table 4 

Means and Standard Deviations of Student Math Growth 

by Teacher Level of Use 
Grades 3-8 



LoU Groups 



III 
IV-A 
IV-B 
V 



N 



12 
18 
1 
1 



Mean 
Growth 



-.2056 
-.3080 
1.5553 
-.8348 



Standard Deviation 



1.1572 
1.9980 
0.0 
0.0 
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Table 5 

Analysis of Variance of Math Growth 
by Teacher Level of Use 



Sources 


SS 


df MS F 


LoU 


.824 


.140* 


Eiror 


35.465 


29 2.947 


Total 


86.289 


31 2.784 


* p > .05 






Table 


6 displays 


the means and standard deviations for 


language growth scores 


*^jr v-ca^iicj. xjcvcx \jl US6. Anaj.ysis Or 


variance results among 


uiit; uiedns in lanyuay e are presented in 


Table 7. 


The results 


of the analysis of variance indicated no 


significant differences 


among the means in language achievement. 






Table 6 


Means 


and Standard 


Deviations of Student Language Growth 




by 


Teacher Level of Use 






Grades 3-8 


LoU Groups 


N 


Mean Standard Deviatior 






Growth 


III 


11 


-.6058 1.9197 


IV-A 


22 


-.3243 1.8092 


IV-B 


1 


-.1048 0.0 


V 


3 


.8972 1.2190 



- 12 - 



ERIC ^ 



Table 7 

Analysis of Variance of Language Growth 
by Teacher Level of Use 



Sources 



SS 



df 



F 



LoU 



4.637 



2 



2.318 



.722* 



Error 



109.220 



34 



3.212 



Total 



113.857 



36 



3.163 



* p > .05 

Question 2. Do high achieving schools use evaluative data more 
effectively or more frequently than lower achieving schools, when 
other important variables are held constant? 

Analysis of LoU in 34 schools which had at least three 
teacher or principal LoU ratings indicated that 10 schools or 29% 
were using evaluation reports at a Mechanical Use level. Eleven 
schools or 32% were using evaluative reports at a Routine level 
and 2 schools or 6% were "enhanced users" of evaluation reports. 

When schools were grouped by achievement growth, analysis 
found individuals at higher achieving schools more frequent users 
of evaluative data. Yet, the analysis of teacher use of reports 
found no differences between low and high achieving schools. 
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Question 3 . If evaluation use makes a difference, what factors 
link it with improved school achievement? 



The researchers hypothesized a positive relationship between 
level of use of evaluation reports and increased student 
achievement. We expected to see different student outcomes at 
different levels of use. For example, LoU III Mechanical Users 
of evaluation reports are struggling with logistics and 
management problems associated with new use of evaluation 
reports. Mechanical users should have less effect on student 
achievement than Routine users. Higher level users, those at LoU 
IV-B Refinement, are adapting their use of evaluation reports to 
increase student iwpac*-. Thus, Refinement users should have more 
positive results than either Mechanical or Routine users. If 
this hypothesj-s held, there would be an increasing relationship 
between level of evaluation use and student achievement. 

A variety of statistical procedures was used to explore the 
relationship between level of use of evaluative data and student 
achievement. Although some relationships were observable within 
schools and grade groups, no significant correlations were 
obtained. The exploratory analysis of student achievement 
suggests that the relationship is not based on the defined level 
of use, but may lie more directly in how the evaluative data are 
actually used in decision making. 



ERIC 
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Limitations 



Several lessons in serendipity were learned during the 
exploration of the relationship of level of evaluation use to 
student achievement. While t! findings did not support the 
hypothesis, the researchers did discover other valuable results. 

First, the anonymity promised to the LoU interviewees proved 
to be a major limitation in the analysis of student achievement 
data. The analysis was also limited by the unavailability of 
student achievement data for approximately half of the teachers. 
We could have screened the teacher sample to ensure the 
availablity of student outcome data. But certainly the small 
sample sizes limited oi r ability to discern postive relationships 
between level of evaluation use and student achievement. 

Second, the innovation of evaluation use might have been 
more explicitly defined before beginning the interviews. While 
the interview cadre did uniformly define the use of PALT 
evaluation reports and the interrater reliability was good, a 
more exhaustive list of the possible patterns of use would have 
helped the early ratings of Level of Use by the inter^/iewers and 
raters. Nevertheless, the LoU interviews did produce an explicit 
list of variations of evaluation use by the end of the study. 



- 15 - 



Finally, further research is needed on the scalar properties 
of the Levels of Use. Some researchers describe it as a nominal 
scale; others view it as an ordinal scale. While it seems 
evident from the definitions of the LoU scale that there are some 
ordinal characteristics, further proof of these characteristics 
would strengthen future research. 

Summary 

The use of evaluation had a large impact on 93% of the 126 
teachers and administrators interviewed during 1986-87. A 
majority of the sample were supporters of PALT evaluation reports 
for use in instructional planning and decision making. 

Research on Levels of Use indicates that it often takes a 
minimum of a year for users of an innovation to move beyond 
Mechanical Use (LoU III) . while PALT reports have been in use 
in Portland for almost ten years, the study found 32% of the 
participants were still at the Mechanical Use level. This 
suggests that with an innovation as complex as evaluation use, it 
can take substantially longer for usors to become comfortable, 
stable Routine users. It is unlikely to occur without inservice 
support to develop knowledge and skills in evaluation use* 

Individuals at the Routine level of evaluation use may or 
may not move to the Refinement level. If they encounter further 
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change or frustration, they may drop hack to Mechanical Use; if 
they receive support and are positively predisposed towards 
evaluation, they may move on to Refinement or Integration. The 
researchers believe that there are personality traits which may 
be a factor in the movement of individuals to higher levels. 
Further evidence for this will be sought in the case studies of 
school evaluation use during 1987-88. 

The link between evaluation use and effective educational 
practice has been well established (Griswold, Cotton, & Hansen, 
1985; Pratt, Winters, & George, 1980; Williams & Bank, 1981). it 
is our belief is that if evaluative data are used routinely, 
appropriately, and effectively for instructional decision making, 
there ought to be observable results in improved student 
performance. The results of this study, while not validating the 
hypothesis, do indicate positive trends in the relationship 
between teacher level of use and student achievement. 

The Level of Use interview technique produces rich 
descriptive information on how individuals use evaluative data in 
instructional planning and decision making. Utilizing this 
theory-based model can guide researchers and practitioners in the 
use of appropriate, timely, and quality evaluation information in 
instructional decision making. 
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APPENDIX 



o Innovation Configuration Ched'list 

o Levels of Evaluation Use Interview Protocol 

o Levels of Use Rating sheet 

o Guide to Level of Evaluation Use 
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INNOVATION CONFIGURATION CHECKLIST 
USE OF PALT EVALUATION REPORTS 



IDEAL USE 



ACCEPTABLE USE 



UNACCEPTABLE USE 



I. Use of PALT Achievement Reports 



Reports are reviewed and 
used extensively for 
Instructional planning 
and decision uaklng. 



Use of PALT Administrative 



Reports are reviewed and 
used extensively for 
Instructional planning 
and decision making. 



3 . Role of Evaluation Reports 



One of multiple 
Influences. 



Some reports are 
emphasized; findings 
used periodically. 



Reports 



Some reports are 
emphasized; findings 
used periodically 
to verify decision. 



Reports are 
skimmed over: 
used infrequently. 



Reports are 
skimmed over; 
used Infrequently. 



in Decision Making 



One of multiple 
Influences . 



Not considered. 



4. Decision Situation? Where Evaluation Is Used 



Instructional planning 
forecasting, goal setting, 
staff development, and 
instructional decisions* 
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Student placement, No clear pattern 
course scheduling, of use. 
monitoring, verifying, 
decisions or actions. 

~^ 



Knows nothing 
about PALT 
achievement 
reports or use. 



Knows nothing of 
administrative 
reports or 
their use. 



Not considered. 



Does not use 
evaluation data 
in instructional 
decision making. 



Ix)U Interview 



0-II/III-VI Are you currently using evaluation reports, specifically 

PALT student achievement or administrative reports? 

If yes, turn page. If no, continue. 



NO 

Have you ever used them in the past? If so, when? \<hy did you stop? 



O/I-II 
I/II 

Knowledge 

Acquiring 
Information 

Knowledge 
Assessing 

Sharing 
Planning 



Final Question 
(Optional) 



If yes, go to PAST USERS (Below) 

If no, continue. 

Have you made a decision to use evaluation reports in the 
future? 

If so, when will you begin use? 

Can you describe your use of evaluation reports for me? 

Are you currently looking for any L^formation about achieve- 
ment or administrative reports? Vvhat kinds? For wlint 
purposes? 

IVhat do you see as the strengths and weaknesses of ?\IT 
evaluation reports in your situation? 

At this point in time, what kinds of questions are you 
asking about using student achievement reports or 
administrative reports? Give examples if necessary. 

Do you ever talk with others and share information about 
PALT evaluation reports? What do you share? 

Vifhat are you planning with respect to using evaluation 
reports? Can you tell me about any preparation or plans 
you^ have been making for the use of PALT student 
achievement or administrative reports? 

Can you summarize for me where you see yourself right now in 
relation to the use of evaluation reports? 



PAST USERS 4" 



Can you describe for me how you organized your use of evaluation reports, 
what problems you found, what the effects appeared to be on students? 

IVhen you assess PALT student achievement and administrative reports at this 
point in time, what do you see as the strengths and weaknesses? 
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A GUIDE TO LEVELS OF EVALUATION USE 
Innovation: Use of Evaluative Data 
(Adapted from Pechman & King, 1986. Hall, 1982 and Hall & Loucks. 1975) 



LoU 0 - NONUSE 

A preliminary stage in which the user is unaware of the potential usefulness of 
evaluative data and its applicability to instructional planning. The nonuser 
does not look for information to support evaluation use. This level is uncommon 
because educators have become more sensitive to the expectation that schools 
should be involved in a continuous evaluation process. 



Decision I .en potential users take action to learn more about evaluative data 
Point I or procedures for usiLg evaluation information, they have moved Ko 
Level I. Orientation . 

LoU I - ORIENTATION 

These users are explorii.g the idea of using PALT reports and are considering how 
the reports might fit taeir own needs for evaluation data. 



Decision When the potential user makes a decision to begin using evaluative 
Point II da^a or reports by a specific time or action, such as asking for a 

consultation with an evaluator to begin using the Student Goal 
Report, tbev have moved to Level II. Prepa ra tion . 

LoU II - PREPARATION 

This is a pre-use level; the person has made the decision to begin using 
evaluation and is preparing for first use. These individuals may recognize the 
need to revi ew evaluation data, but not knowing quite how to do it, they assign 
the task to a staff person. They are not quite certain what to expect, but 
vaguely expect feedback from the designated person or evaluator. 



Decision As users begins first use of evaluation reports by reviewing the 
Point III data and making a serious effort to use data in student placement, 
planning or individualizing instruction, informally or simply, :hey 
have moved to Level III. Mechanical Use. 

LoU III - MiiClIANICAL USE 

At this stage, early users ma> request assistance in understanding and inter- 
preting evaluative data and reports. They may discuss school or classroom 
evaluation information in staff/team meetings or in instructional planning. The 
mechanical user employs evaluation in obligatory ways, e.g., reviewing reports 
in meetings or distributing summaries of key data to staff. The user does not 
consider the quality of the evaluation process and pays little attention to the 
role of evaluation and its use in program implementation, planning, or change. 

Decision When the user has established a comfortable use of evaluative data 
Point IVa within their situation, tbev have moved to Level IV-A. Routine. 



LoU IVa - ROUTINE 

At this stage, regular use of evaluation reports as part of the annual cycle of 
program planning, forecasting, and assessment is iu place. The users routinely 
employs trend data in program planning. A skeleton evaluation process is 
implemented; routine users do not implement a systematic organizational use of 
the evaluation reports, but use evaluative data in limited, nondynamic ways. 

Decision At this level, users begin to make small changes in their use of 
Point IVb data on the basis of formal or informal evaluation information to 
improve their u se and increase student outcomes. 

LoU IVb - REFINEMENT 

The Refinement level marks the beginning of systematic use of evaluative data to 
make program changes. At this stage, the use of evaluation reports is 
restricted. There may be attempts to mold report data to provide information 
needed for specific school or classroom decisions. Users might create special 
report summaries tailored to meet their needs. Decision makers seek out usable 
data by modifying the existing reports or providing staff training to assure 
that the data are systematically generated and dependably reported. The user's 
commitment to use of evaluation data is put into action; this is the start of 
turning theory into practice. 



Decision By initiating changes in the use of evaluative da^^a in 
Point V coordination with what colleagues are doing, the user reaches Level 
V, Integration. The decision to take action and make instructional 
changes is based on informal feedback from the evaluator, formal 
test data, and evaluation reports. 

LoU V - INTEGRATION 

At this level, the user implements higher levels of use by proactively 
supporting the use of evaluation information both verbally and through 
managerial actions. The systematic planned use of evaluation is well 
coordinated. The user carefully determines what types of evaluative data have 
practical use for different audiences. Test data are routinely disseminated and 
responded to by staff; colleagues collaborate to assure the report information 
is interpreted, understood, and disseminated to staff. Instructional changes 
are monitored and where possible, are based on evaluation report findings. 



Decision User begins exploring major modifications or replacement of 
PohiC VI nnrr ent evaluation data and reports. 



LoU VI - RENEWAL 

This user reevaluates the quality of evaluative data to assure it adequately 
serve her needs for instructional planning. This review process is continuous 
and well integrated into the school structure. It is comprehensive and reaches 
all levels of administration, staff, and parents. Renewal decision makers 
routinely expect student achievement data, take action to obtain it, analyze its 
worth and practicality, and use it to make data based instructional decisions. 
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